Dynamic management of multiple persistent data stores

ABSTRACT

Dynamic management of multiple persistent data stores is disclosed. One or more data objects are associated with a client context, e.g., an instance of a client application program. The one or more data objects are retrieved from one or more of a plurality of persistent data stores and provided to the client context in a manner such that the one or more data objects appear to the client context to come from a single source, even if in fact the objects have been retrieved from two or more different persistent stores.

CROSS REFERENCE TO OTHER APPLICATIONS

U.S. patent application Ser. No. 11/104,187, entitled MANAGINGCROSS-STORE RELATIONSHIPS TO DATA OBJECTS, filed concurrently herewith,is incorporated herein by reference for all purposes.

U.S. Pat. No. 5,873,093 entitled METHOD AND APPARATUS FOR MAPPINGOBJECTS TO A DATA SOURCE filed May 28, 1997, is incorporated herein byreference for all purposes; U.S. Pat. No. 5,898,871 entitled DYNAMICOBJECT COMMUNICATION PROTOCOL filed May 27, 1997, is incorporated hereinby reference for all purposes; U.S. Pat. No. 5,956,728 entitled OBJECTGRAPH EDITING CONTEXT AND METHODS OF USE filed Jul. 17, 1996, isincorporated herein by reference for all purposes; and U.S. Pat. No.6,085,197 entitled OBJECT GRAPH EDITING CONTEXT AND METHODS OF USE filedOct. 6, 1998, is incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

Traditional implementations of object-storage mapping frameworks focuson enterprise client/server databases and only allow for a fixedconfiguration of databases or persistent data stores. Such aclient/server enterprise environment does not support object-storagemapping in an environment in which the set of data stores are dynamicsuch as in a desktop environment where the set of available data sourcesmay change often as file systems and network resources are dynamicallydetected and integrated into a user's computing environment. Prior artimplementations do not allow the dynamic addition or removal ofpersistent data stores and do not provide a corresponding synchronousgrowth or reduction of the visible data set available to a clientcontext as data stores are added or removed, respectively. Suchfunctionality is desirable when the set of data stores is possiblydynamic.

Thus, there is a need for the dynamic management of persistent datastores.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 illustrates an embodiment of a persistent store coordinatorconfigured to manage dynamically multiple persistent data stores.

FIG. 2 illustrates an embodiment of an environment in which the storeobjects associated with different persistent store coordinators sharethe same persistent data sources.

FIG. 3 illustrates an embodiment of an object graph associated with aclient context.

FIG. 4 illustrates an embodiment of the components of a model that isused to formally describe a particular object (or object graph) inmemory.

FIG. 5 illustrates an embodiment of a model that formally describes theobject graph depicted in FIG. 3.

FIG. 6 illustrates an embodiment of a process used when an applicationis launched and a client context of the application is initialized.

FIG. 7 illustrates an embodiment of processes performed by a persistentstore coordinator.

FIG. 8 illustrates an embodiment of a process used to add a persistentdata store.

FIG. 9 illustrates an embodiment of a process used to insert a newobject description into the model associated with a persistent storecoordinator.

FIG. 10 illustrates an embodiment of a process used to retrieve (i.e.fetch) data from one or more persistent data stores.

FIG. 11 illustrates an embodiment of a process used to retrieve datafrom multiple persistent data stores, if applicable.

FIG. 12 illustrates an embodiment of a process used to save an object toone or more persistent data store(s) data associated with an object inmemory.

FIG. 13 illustrates an embodiment of a process used to save an object inmemory to a primary store and a mirror store.

FIG. 14 illustrates an embodiment of a process used to migrate objectsfrom one store to another.

FIG. 15 illustrates an embodiment of a process used to cache retrieveddata and service client requests using cached copies, if possible.

FIG. 16 illustrates an embodiment of a process used to purge cachedcopies of objects that are no longer in use by any clients.

FIG. 17 illustrates an embodiment of a process used to synchronize acached copy with its associated persisted data, if appropriate.

FIG. 18 illustrates an embodiment of an environment in which thepersistent store coordinator manages communication between a clientcontext and available persistent data stores, one of which is an“in-memory” store.

FIG. 19 illustrates an embodiment of a process used to add a store,including an in-memory store.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess, an apparatus, a system, a composition of matter, a computerreadable medium such as a computer readable storage medium or a computernetwork wherein program instructions are sent over optical or electroniccommunication links. In this specification, these implementations, orany other form that the invention may take, may be referred to astechniques. A component such as a processor or a memory described asbeing configured to perform a task includes both a general componentthat is temporarily configured to perform the task at a given time or aspecific component that is manufactured to perform the task. In general,the order of the steps of disclosed processes may be altered within thescope of the invention.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Dynamic management of multiple persistent data stores is disclosed. Oneor more data objects are associated with a client context, e.g., aninstance of a client application program. The one or more data objectsare retrieved from one or more of a plurality of persistent data storesand provided to the client context in a manner such that the one or moredata objects appear to the client context to come from a single source,even if in fact the objects have been retrieved from two or moredifferent persistent stores. In an embodiment, a persistent storecoordinator is employed to manage a plurality of persistent data stores.The persistent store coordinator allows the dynamic addition or removalof the persistent data stores and grows or shrinks the visible data setsof one or more client contexts accordingly. Communication between clientcontexts and the persistent data stores is facilitated by thecoordinator in a manner in which the data associated with the individualpersistent data stores appear to be associated with a single data sourceto the client contexts. Thus, the persistent store coordinator providesto the client contexts a unified façade of all the data stored in theavailable persistent data stores.

FIG. 1 illustrates an embodiment of a persistent store coordinatorconfigured to manage dynamically multiple persistent data stores. In theexample shown in environment 100, persistent store coordinator 102 usesstore objects 106 and 108 which represent a pair of persistent datastores, structured query language (SQL) database 110 and extensiblemarkup language (XML) file 112, respectively, to make objects stored indatabase 110 and file 112 available to a plurality of client contexts,such as client context 104. Each of the client contexts manages anobject graph that represents the objects in memory that are in use bythat client context and the relationships between related objects. Asdepicted in FIG. 1, client context 104 tracks the objects it is usingand has stored in memory via object graph 114.

The object graphs of the client contexts are each formally described byone or more models (i.e. one or more forms of data files). Persistentstore coordinator 102 realizes a single model 116 based on the summationof model(s) associated with the object graphs of the client contexts.Model 116 serves as a shared means for communication between the clientcontexts, such as client context 104, and persistent store coordinator102. The specific model associated with a particular object in memory(which is part of model 116) enables persistent store coordinator 102 toinfer the manner in which that particular object in memory maps into thestructure of an associated persistent data store. The mapping associatedwith each type of object in memory can be inferred programmatically bymodel 116 and/or, in some embodiments, configured manually, e.g., by adeveloper of an application associated with client context 104. For thesame object in memory, different mappings may exist for different typesof persistent data stores. Based on the associated mapping, an objectmay be mapped into one or more tables in a database or similarly intoother storage formats associated with other kinds of persistent datastores. Likewise, individual attributes of an object may be mapped toone or more fields in a database or similarly to other storage formatsassociated with other kinds of persistent data stores, and the type ofan attribute may be switched (e.g., a number might be stored as a stringor vice versa). That is, the mapping provides the translations forattributes, types, and locations of the data elements that comprise anobject. Upon realization of model 116, persistent store coordinator 102associates with each type of store object its corresponding mapping. Asdepicted in FIG. 1, mapping 118 is associated with store object 106, andmapping 120 is associated with store object 108. In some embodiments,when a store is added, as described more fully below, a client contextthat wants to access data objects associated with the new store providesto persistent store coordinator 102 “configuration” data or otherinformation that identifies to the persistent store coordinator 102 theobject(s) within the model 116 in connection with which the clientcontext expects to use the new store. The persistent store coordinator102 uses this configuration information and metadata and/or other dataassociated with the new store to infer a mapping between the objectsidentified in the configuration data and the corresponding elements inthe new store. For example, if a client context were to indicate itdesired to use a new store in connection with its “person” and “phone”objects, the persistent store coordinator 102 might infer in oneembodiment that a data element referred to as “name” in the new storemaps to “person” and a “primary phone” element maps to “phone”.

In the example shown, each store object, such as store objects 106 and108, represents an external persistent data store, e.g., an SQLdatabase, XML file, relational database management system (RDBMS),binary file, etc. A persistent data store may be located on a harddrive, on a removable storage media, on a server or workstation in anetwork, etc. Each store object handles access to one type of file,e.g., store object 106 represents SQL database 110 and store object 108represents XML file 112. In some embodiments, a persistent storecoordinator (PSC) such as PSC 102 may be associated with one or moreclient contexts and one or more store objects, each store object beingassociated with a single persistent data store. In some embodiments,more than one PSC may access the same persistent data store, but eachuses its own associated store object to do so. In some embodiments, astore object encapsulates a concrete file format access mechanism andhas an application program interface (API) that enables communicationwith its associated persistent data store. The mapping associated witheach store identifies the manner in which object-storage mapping isimplemented, i.e., the manner in which objects in memory are mapped intoexternal data stored in persistent data stores and, likewise, the mannerin which external data in persistent data stores are mapped into objectsin memory. One manner in which objects are mapped to a data source isdescribed in U.S. Pat. No. 5,873,093 entitled METHOD AND APPARATUS FORMAPPING OBJECTS TO A DATA SOURCE issued Feb. 16, 1999, which wasincorporated herein by reference above.

The client contexts, such as client context 104, may each correspond toan individual window of an application. However, the individual clientcontexts may be configured in a variety of ways. That is, a persistentstore coordinator such as persistent store coordinator 102 can have anarbitrary number of possibly dissimilar clients. In some embodiments,multiple client contexts may communicate with the same persistent storecoordinator, but each client context exclusively communicates with asingle persistent store coordinator. Client contexts may be nested intheir own hierarchy with parent client contexts serving to delimitatethe scope of changes sub-contexts (or child client contexts) make. Eachstore object is managed by a single persistent store coordinator. Aparticular application or process may have multiple persistent storecoordinators.

FIG. 2 illustrates an embodiment of an environment in which the storeobjects associated with different persistent store coordinators sharethe same persistent data sources. Specifically, environment 200 includesthree distinct persistent store coordinators, 202, 204, and 206, whichare independent of one another. In some embodiments, each persistentstore coordinator is part of a “stack” which includes the persistentstore coordinator and associated client context(s) and store object(s).An application or process may be characterized by multiple stacksdepending upon the requirements of the client contexts. In the exampleshown in FIG. 2, three stacks corresponding to the three persistentstore coordinators are depicted. The object graphs, models, and mappingsassociated with the client contexts, persistent store coordinators, andstore objects, respectively, as depicted in FIG. 1, have been omittedfrom FIG. 2 for conciseness.

As depicted in FIG. 2, each of the client contexts 208-216 is distinct(i.e. each client is associated with a single persistent storecoordinator), and the client contexts may be associated with the sameapplication or different applications. Similarly, each distinct storeobject 218-224 is managed by a single persistent store coordinator.However, multiple store objects, each of which is associated with adifferent persistent store coordinator, may represent the same externalpersistent data store and even the same data file. For instance, asdepicted in FIG. 2, store objects 218 and 220 both represent SQLdatabase 226 and store objects 222 and 224 both represent XML file 228.Thus, by employing multiple stacks, concurrent requests to eachpersistent data store are possible. In an embodiment in which multiplePSC's access the same persistent data store the PSC's may be configuredto mediate conflicts between them automatically. Likewise, if a PSCaccesses the same persistent data store for two or more client contexts,the PSC may be configured to resolve conflicts on behalf of the clientcontexts or defer conflict resolution to the client contexts. In someembodiments, mechanisms are provided for developers of the clientcontexts to handle conflicts in the most appropriate manner to fit theneeds of their application. In some embodiments, conflicts resultingfrom multiple changes to data may be resolved by selecting one set ofchanges and discarding the rest (e.g., the first change or last changemay be selected). In some embodiments, multiple changes are merged.

FIG. 3 illustrates an embodiment of an object graph associated with aclient context. In some embodiments, object graph 300 corresponds toobject graph 114 of FIG. 1. Each client context manages an object graph,such as object graph 300, that represents the objects in memory that arein use by that client context. An object graph also provides therelationships between related objects. In some embodiments, aprogramming interface, such as a “container”, is used by the clientcontext to refer to the objects and the corresponding object graphassociated with the client. In such embodiments, the objects of theclient context are created in, run from, and managed by the container.In FIG. 3, object graph 300 includes three objects: a person object 302,an address object 304, and a phone number object 306. Object graph 300also depicts the relationships between these three objects.Specifically, each person object 302 has an associated address object304 and an associated phone number object 306. An object graph, such asobject graph 300, is used by a client context to keep track of theobjects it is using and has stored in memory and the relationshipsbetween related objects. As described above, in some embodiments aformal description of the object graph of a client context is providedto an associated persistent store coordinator via one or more modelsthat describe the objects that comprise the object graph.

FIG. 4 illustrates an embodiment of the components of a model that isused to formally describe a particular object (or object graph) inmemory. As depicted in FIG. 4, a model includes for each type of objectin an object graph associated with a client context with which a PSC isassociated an entity 400. Each entity includes in the example shown anentity name 402, attribute(s) 404, relationship(s) 406, and optional an“other” component 408. An entity maps to an object class and describes aparticular type of object. For example, a person entity exists for aperson object, an address entity exists for an address object, etc. Insome embodiments, a model exists for each type of object (i.e. entity)in memory. In some embodiments, a model includes more than one entity.An entity 400 includes one or more attributes 404 that describe theentity 400. One or more relationships 406 describe links (i.e. pointers)between related entities of a model. For example, a “person” entity mayinclude a relationship indicating that each person object has associatedwith it an address object. The relationship may indicate a “target” or“destination” entity associated with the other end of the relationship(e.g., a relationship reflecting that each “person” object has anassociated “address” object may point in the model to an “address”entity); a way in which the particular target object related to aparticular instance of an object associated with the entity may beidentified, such as a pointer, key, or other data to be used to identifythe target data; and/or other information associated with therelationship. “Other” component(s) 408 may describe other types ofproperties and/or relationships of one or more entities that a modelcorresponds to. In some embodiments, “other” 408 corresponds to “fetchedproperties”, a loosely defined relationship to a target data objectwhich is described in detail in co-pending U.S. patent application Ser.No. 11/104,187 entitled MANAGING CROSS-STORE RELATIONSHIPS TO DATAOBJECTS filed concurrently herewith, which was incorporated herein byreference for all purposes above.

FIG. 5 illustrates an embodiment of a model that formally describes theobject graph depicted in FIG. 3. The object graph of FIG. 3 includesthree types of objects: a person object 302, an address object 304, anda phone number object 306. In some embodiments, each type of object isrepresented in a model by an associated entity. In model 500, a personentity 502 represents objects of the type (or class, e.g.,) “person”, anaddress entity 504 represents objects of the type “address”, and a phonenumber entity 506 represents objects of the type “phone number”. Personentity 502 includes a name 508 and attributes 510: <name> and <jobtitle>. Person entity 502 also describes relationships 512 to addressentity 504 and phone number entity 506. In some embodiments, therelationships 512 may include a description of how the target dataobject related to a particular instance of a person object is found,e.g., the key or other data used to access the target data in a datastore associated with the target object. No “other” components (i.e. 408of FIG. 4) are associated with person entity 502 in the example shown.Address entity 504 includes an entity name 514 and attributes 516:<number>, <street>, <city>, <state>, and <zip>. No relationships or“other” components (i.e. 406 and 408, respectively, of FIG. 4) areassociated with address entity 504. Phone number entity 506 includes anentity name 518 and attributes 520: <area code> and <number>. Norelationships or “other” components (i.e. 406 and 408, respectively, ofFIG. 4) are associated with phone number entity 506.

The entire object graph 300 of FIG. 3 is formally described by model500, which includes the person entity 502, address entity 504, and phonenumber entity 506. One or more models, such as model 500, representingone or more object graphs associated with one or more client contextsare employed by a persistent store coordinator, such as persistent storecoordinator 102 of FIG. 1, to realize a single, composite model, such asmodel 116 of FIG. 1, that enables the persistent store coordinator andassociated client contexts to communicate.

In some embodiments, an object and its associated data file stored in apersistent data store may include only a subset of a model. A subset ofa model, referred to herein sometimes as a “configuration”, defines asubset of one or more entities of the model that is/are associated withan external data file or other persistent data store. For example, model500 of FIG. 5 includes the person, address, and phone number entitiesand a particular configuration, e.g., including the person and addressentities, may be associated with a particular persistent data store,such as might occur if names and addresses were stored together in onepersistent store but phone numbers were stored in another, separatestore.

FIG. 6 illustrates an embodiment of a process used when an applicationis launched and a client context of the application is initialized. Inthe example shown, at 602 a persistent store coordinator object isinstantiated by the client context and/or associated application. Insome embodiments, a new persistent store coordinator object is notinstantiated at 602, but rather, a pre-existing persistent storecoordinator object is selected to serve the initialized client context.At 604, the persistent store coordinator associated with the initializedclient context is provided a model definition that formally describesthe object graph managed by the client context in memory. In someembodiments, the model definition is dynamically generated when anapplication is launched or a client context is initialized.Alternatively, the initialized client context may already include apre-existing model definition (e.g., if a model definition was generatedduring a previous run of the client context or defined or otherwiseconfigured statically by a developer of the associated clientapplication). At 606, the client context designates to its associatedpersistent store coordinator one or more persistent data stores itexpects to interact with, e.g., save data to, retrieve or fetch datafrom, query, etc. In addition, as depicted in FIG. 6, at 606 the clientcontext also designates one or more objects to be populated from thedesignated stores. In an embodiment, 606 includes an identification of a“configuration” that includes and/or describes the entities associatedwith the client context that are to be populated using the designatedstore. At 608, the data files (i.e. objects) requested from thedesignated stores in 606 are received by the client context from thepersistent store coordinator.

FIG. 7 illustrates an embodiment of processes performed by a persistentstore coordinator. At 702, a persistent store coordinator receives amodel definition and builds (or adds to) a model based on the modeldefinition. In some embodiments, the model definition received at 702 isthe model definition provided in 604 of FIG. 6. In some embodiments, themodel definition includes one or more models that define the objectgraphs associated with one or more client contexts associated with thepersistent store coordinator. If multiple models exist, the persistentstore coordinator utilizes the union of all the models to realize orbuild a single model that allows the persistent store coordinator toservice its associated client contexts. In some embodiments, model 116of FIG. 1 is the model built by the persistent store coordinator at 702.

In some embodiments, once a persistent store coordinator is instantiatedand a model realized by the persistent store coordinator, the persistentstore coordinator exclusively manages data communication between itsassociated client contexts and the available persistent data stores. Forexample, stores are dynamically added to and/or removed from the set ofavailable persistent data stores by the persistent store coordinator asand when requested by one or more client contexts associated with thepersistent store coordinator (704). If a persistent data store becomesunexpectedly unavailable (e.g., if a removable media store isunexpectedly removed, a server in the network goes down, etc.), theobject graphs of the affected client contexts are pruned by thepersistent store coordinator (706) so that the object graphs alwaysreflect only the data (i.e. objects) that exist in stores currentlyavailable to the persistent store coordinator. For example, if theperson and address objects of object graph 300 of FIG. 3 are externallystored in a XML file and the phone number object is externally stored ina SQL database and the SQL database is unexpectedly removed, the phonenumber object is pruned from the object graph. In some embodiments, 706includes informing each affected client context that a store has becomeunavailable and/or informing each affected client context that specifiedaffected objects should be pruned from the object graph associated withthe client context. In some embodiments, the persistent storecoordinator informs the client contexts which stores have becomeunavailable, and the client contexts eliminate references to objectsloaded from those stores.

In some embodiments, if a persistent data store that previously becameunexpectedly unavailable becomes available again, the persistent storecoordinator ensures that the object graphs of the affected clients areupdated, i.e. the affected objects are re-inserted into the objectgraphs. For the example given above, if the SQL database that storesphone numbers becomes available again, the phone number object isre-inserted into the object graph from which it was pruned when the SQLdatabase became unexpectedly unavailable. The persistent storecoordinator inserts descriptions of new objects into its model andremoves descriptions of objects which are no longer in use from itsmodel as and when it becomes aware of such changes (708). In someembodiments, 708 includes receiving requests and/or updates from one ormore client contexts about the object(s) with which the client contextdesires to work. The persistent store coordinator receives and servicesrequests from one or more associated client contexts to access one ormore available stores (e.g. to save data to, fetch data from, query,etc.) (710). In some embodiments, such requests are serially handled bythe persistent store coordinator. 704-710 may be performed by thepersistent store coordinator any number of times and in any order and/orsimultaneously.

In some embodiments, the persistent store coordinator represents asingle access point to multiple persistent data stores. The persistentstore coordinator represents multiple stores as a union to itsassociated client contexts. That is, all external data files frommultiple persistent data stores appear to be associated with a datasource to the client contexts. The dynamic removal and/or addition ofpersistent data stores is supported by the persistent store coordinatorand conveyed by the persistent store coordinator to the client contextsby the synchronous update of affected object graphs (i.e. by pruningand/or inserting objects) of the client contexts in memory. The clientcontexts are completely isolated from the external persistent datastores and do not have to be concerned about the structure in whichobjects are persistently stored. In addition, the persistent storecoordinator may provide client contexts with additional data handlingfunctionalities. For example, the persistent store coordinator mayvalidate data, guarantee integrity, ensure relationships, performconversions, etc., thus relieving the client contexts of the overhead ofperforming such operations individually.

In some embodiments, 704 is performed each time a new store becomesavailable. In some embodiments, the PSC is informed when a new storebecomes available (e.g., a system connects to a network or a volume ofremovable media is inserted into a drive) and automatically accesses thestore if the PSC determines, e.g., by “sniffing” metadata or other dataassociated with the store, that the store is or may be of interest toone or more client contexts with which the PSC is associated. In someembodiments, the client context becomes aware of a new store becomingavailable and instructs the PSC to add the store as one to which theclient context has access.

FIG. 8 illustrates an embodiment of a process used to add a persistentdata store. In some embodiments, process 800 is used to implement 704 ofFIG. 7. At 802, the persistent store coordinator receives a request toadd a new store by a client context associated with the persistent storecoordinator. In some embodiments, the request by a client context to itsassociated persistent store coordinator to add a particular persistentdata store includes the location of the store, the configurations ofobjects it would like to populate using the store, and/or the type ofthe store. The location of the store may be in the form of a uniformresource identifier (URI), directory, etc. In some embodiments, theconfigurations of one or more objects a client context wishes topopulate from the added data store can be determined by the clientcontext by “sniffing” the metadata in the associated external datafiles. In some embodiments, the reading (e.g. “sniffing”) and/or writingof metadata of a data file by a client context does not requirefacilitation by a persistent store coordinator (which is required whenthe actual data of the data file is read or written) and does not dependupon the type of data stored in the data file. Thus, if theconfiguration of an object a client context would like to populate usingthe store is not directly known to the client context, the metadata ofthe associated data file can be “sniffed” to determine theconfiguration. Likewise, the type of the store, i.e. the format in whichdata is stored in the store, (e.g., SQL, RDBMS, XML, binary, etc.), maybe either explicitly supplied by the client context if it is known ormay be “sniffed” from the metadata of a data file corresponding to anobject the client context would like to populate using the store. Insome embodiments, the type and/or other data as described above is“sniffed” by the persistent store coordinator. In some embodiments, thetype of the store is implied by the external store representation ondisk, and the type of the store is merged into the metadata, e.g., bythe persistent store coordinator, as “permanent implied metadata”.

At 804, the persistent store coordinator instantiates a store object torepresent the new store added in 802 and associates a mapping with thestore. In some embodiments, the persistent store coordinator uses thetype of the store to instantiate an appropriate store object and realizean appropriate mapping for the added store. For example, if the addedstore is a SQL database, an object from the SQL store class isinstantiated by the persistent store coordinator at 804; if the addedstore in an XML file, an object from the XML store class is instantiatedby the persistent store coordinator at 804. The store objectinstantiated at 804 directly interacts with and facilitates datatransfers between the persistent data store added at 802 and thepersistent store coordinator. Also at 804, the persistent storecoordinator associates with the instantiated store object a mapping thatspecifies the manner in which data stored in the persistent data storeadded at 802 maps into one or more associated objects in memory. In someembodiments, the type of a persistent data store is used by thepersistent store coordinator to determine the mapping associated withthat store. The type (or mapping) may be explicitly supplied by theclient context or may be dynamically generated by “sniffing” themetadata associated with a data file of the store and/or otherwiseprogrammatically inferring the mapping. In some embodiments, the typeand/or mapping is automatically deduced by the persistent storecoordinator whenever possible.

FIG. 9 illustrates an embodiment of a process used to insert a newobject description into the model associated with a persistent storecoordinator. In some embodiments, process 900 is used to implement 708of FIG. 7. At 902, the persistent store coordinator receives adescription of a new object added to the object graph of a clientcontext associated with the persistent store coordinator. In someembodiments, the description of a new object received at 902 may resultfrom the addition of a new client context to the persistent storecoordinator. The description received at 902 includes a description ofthe type of object, e.g., a description of an entity such as thosedescribed above in connection with FIGS. 3-5. In some embodiments, thedescription received at 902 also includes the destination store of thenew object. The destination store may be specified by a URI. If thedestination store is omitted from the description of the new objectreceived at 902, any store that can compatibly store the new object isselected by the persistent store coordinator as the destination store.In some embodiments, the first store that can persist an object of thattype is selected by the persistent store coordinator as the destinationstore. At 904, the description of the new object received at 902 isadded as an entity in the model associated with the persistent storecoordinator.

In some embodiments, objects to be fetched from persistent data storesare described by a query language, i.e. “predicates”, that isindependent of the concrete storage mechanism (which might be SQL, XML,binary data, etc.) and gets translated by the persistent storecoordinator and the store objects into the appropriate query languagefor each persistent data store format.

FIG. 10 illustrates an embodiment of a process used to retrieve (i.e.fetch) data from one or more persistent data stores. In someembodiments, process 1000 is used to implement 710 of FIG. 7. In someembodiments, the process of FIG. 10 is implemented on a PSC. At 1002, arequest from client context to populate one or more objects with data isreceived. In some embodiments, the request received at 1002 specifiesthe entity of the objects the client context desires to populate. Insome embodiments, the request received at 1002 specifies one or moresource stores to retrieve data from. In some embodiments, if the sourcestore is not specified in the request received at 1002, the persistentstore coordinator determines programmatically which stores to search. Insome embodiments, the persistent store coordinator retrieves data fromall available stores that have data associated with an entity associatedwith the request, determined for example based at least in part onconfiguration information received previously from the requesting clientcontext indicating which entities are associated with each data store.At 1004, the persistent store coordinator uses the store object(s) ofthe source store(s) to retrieve data from the associated store(s). At1006, the persistent store coordinator provides the retrieved data tothe requesting client context. In some embodiments, the data is providedto the client context as if it originates from a single source,regardless of how many persistent stores were accessed to retrieve thedata.

FIG. 11 illustrates an embodiment of a process used to retrieve datafrom multiple persistent data stores, if applicable. In someembodiments, the process of FIG. 11 is used to implement 1006 of FIG.10. As depicted in FIG. 11, at 1102 data retrieved from multiple storesis aggregated, if applicable. At 1104, the requested data is provided tothe client context such that all data appears to the client context tocome from a single source. Aggregating like data (i.e. data thatcorresponds to a particular entity) retrieved, where applicable, frommultiple stores, sometimes referred to herein as “striping”, allows theretrieval in a single operation of all the objects associated with anentity from all available and applicable persistent data stores. Such anapproach enables the developer of the client context (e.g., applicationprogram) to look to the PSC as a single data source from whichpersistently stored data can be retrieved using a single interface,without requiring the developer to anticipate which types of data storemay be used to persistently store data associated with the clientcontext and without requiring the developer to know or write code toenable the client context to access directly the persistent data storethat sits behind the PSC.

FIG. 12 illustrates an embodiment of a process used to save an object toone or more persistent data store(s) data associated with an object inmemory. In some embodiments, process 1200 is used to implement 710 ofFIG. 7. At 1202, the persistent store coordinator receives a requestfrom an associated client context to save an object in memory to one ormore persistent data store(s). The request received at 1202 specifiesthe entity of the object the client context desires to save and the datato be saved. In some embodiments, the request received at 1202 specifiesone or more destination store(s) to save the data associated with theobject to. In some embodiments, the entity in the model of thepersistent store coordinator that corresponds to the object the clientcontext desires to save to persistent storage is associated with one ormore destination store(s). If a destination store is not specified by aclient context either when inserting an entity associated with a newtype of object into the model of the persistent store coordinator (e.g.,at 902 of FIG. 9) or when actually requesting an object to be saved(e.g., in 1202), in some embodiments, the persistent store coordinatorsaves the object to any available persistent data store that supportsthe data associated with the object. In some embodiments, the persistentstore coordinator saves the object to the first store that can persistan object of that type. At 1204, the persistent store coordinator usesthe store object(s) of the destination store(s) to save the data to theassociated persistent data store(s). In some embodiments 1206 includesinforming the client context when the save request is completed.

In some embodiments, the persistent store coordinator provides a“mirroring” functionality for objects stored in the persistent datastores. If a client context desires to save an instance or element of anobject or entity at a primary store and also at a backup (i.e. mirror)store, the client can either specify the primary and mirror destinationstores in a save request (e.g., 1202 of FIG. 12) or the information canbe encapsulated within the model associated with the persistent storecoordinator (e.g., at 902 of FIG. 9) and/or determined programmaticallyby the PSC. The persistent store coordinator subsequently remembers thatthe data associated with an object that is stored in primary and backupstores are mirrors of each other and that there is only one objectassociated with the two sets of data. The persistent store coordinatorensures that any changes saved to a mirrored object are saved at boththe primary and mirror stores, i.e. the persistent store coordinatorensures that the data sets in the primary and mirror stores are alwayskept in sync with one another. When mirrored data is retrieved frompersistent storage, the persistent store coordinator returns to a clientcontext only a single object.

FIG. 13 illustrates an embodiment of a process used to save an object inmemory to a primary store and a mirror store. In some embodiments, theprocess of FIG. 13 is implemented on a PSC configured to “mirror” dataobjects. In some embodiments, mirroring may be requested and/orspecified for one or more elements of an object by a client context.Process 1300 starts at 1302 with the first element associated with anobject for which a save request has been received. In some embodiments,a save request may apply to one or more objects of a type specified inthe save request, in which case each individual instance of the objectcomprises one or more “elements” as referred to in FIG. 13. In someembodiments, an instance of an object may include more than one elementof data, and each such element is treated as an “element” as that termis used in FIG. 13. At 1304 it is determined whether the current elementwas previously mirrored. In some embodiments, the model associated withthe persistent store coordinator includes information pertaining towhether an element was previously mirrored (e.g., if the element isassociated with more than one destination store). If it is determined at1304 that the current element was not previously mirrored, it isdetermined at 1306 whether mirroring of the current element is requestedwith the current save. If it is determined at 1304 that the currentelement was previously mirrored or it is determined at 1306 thatmirroring is requested for the current element with the current save,then at 1308 the persistent store coordinator uses the associated storeobject(s) to save the current element to both the primary and mirrorstores. If it is determined at 1306 that mirroring of the currentelement is not requested, at 1310 the current element is saved to itsassociated destination store. At 1312 it is determined whether allelements associated with the save request have been saved. If it isdetermined at 1312 that all elements have not been saved, then the nextelement is selected at 1314, and process 1300 resumes from 1304. If itis determined at 1312 that all elements associated with the request havebeen saved, process 1300 ends at 1316.

In some embodiments, the persistent store coordinator provides means to“migrate” data from one persistent data store to another store which maybe of the same type or a different type. Migration allows the conversionof data in a store from one type or format to another or the creation ofa duplicate copy of the data in a store that is of the same type as thesource store. For example, all the data stored in a SQL database can beexported to an XML file, to another SQL database, or to any other typeof available persistent data store. Migration may be requested by one ormore client contexts. However, since the client contexts are completelyisolated from the manner in which data is stored in persistent storage,a migration operation in some embodiments is performed in a manner suchthat the move is transparent to affected objects and object graphsassociated with the client contexts that use the migrated data. That is,the client contexts can continue working with their objects and objectgraphs even when the data underlying one or more of the objects is inthe process of moving or has moved to a different location. Thepersistent store coordinator facilitates and tracks all data moves ormigrations within and/or among the persistent data stores and updatesthe pointers for all affected objects in memory, e.g., by updating amodel and/or one or more mappings associated with the PSC and theaffected data and/or objects, so that objects in memory are alwaysmapped to the correct persistent data store.

FIG. 14 illustrates an embodiment of a process used to migrate objectsfrom one store to another. In some embodiments, the process of FIG. 14is implemented by a PSC. At 1402, a request is received from anassociated client context to migrate one or more objects(s) from onestore to another. The request received at 1402 includes the source storeand the destination store, both of which may be specified by a URI. Insome embodiments, the request received at 1402 also includes thedestination store type. If the destination store type is not specifiedin the request received at 1402, it can be dynamically determined by thepersistent store coordinator, e.g., by “sniffing” into the metadata of afile associated with the destination store. At 1404, a destination storeis added if it is required (e.g., if the request of 1402 included arequest to add a specific destination store). In some embodiments, 1404includes instantiating a store object for the destination store. At1406, the store object(s) associated with the source and destinationstores are used to move data from the source store to the destinationstore. Although the operation of client contexts and their associatedobjects is not affected by the migration of process 1400, in someembodiments, the persistent store coordinator informs any clients usingany affected objects of the new location of the objects in persistentstorage at 1408. A new location may be specified by a URI. 1408 ofprocess 1400 is useful for a client context if it ever needs to specifyduring a subsequent operation (e.g., saving data, retrieving data, etc.)a source and/or destination store which has been affected by themigration of process 1400.

In some embodiments, a persistent store coordinator caches data eachtime a client retrieves an object from a persistent data store. FIG. 15illustrates an embodiment of a process used to cache retrieved data andservice client requests using cached copies, if possible. At 1502, arequest to retrieve or fetch an object is received from an associatedclient context. It is determined at 1504 whether the object requested in1502 is in cache. If it is determined at 1504 that the object requestedin 1502 is in cache, the request of 1502 is serviced at 1506 using thecached copy. In some embodiments, a reference counter is employed foreach object stored in cache to determine whether any client contexts areusing the object. Thus, upon servicing a request using a cached copy at1506, the reference counter associated with the requested object isincremented at 1508. If it is determined at 1504 that the objectrequested in 1502 is not in cache, at 1510 the store object(s)associated with the source store(s) associated with the requested objectare used to retrieve data from the associated persistent data store(s).At 1512 the data retrieved at 1510 is provided to the requesting client.At 1514 the persistent store coordinator adds a copy of the objectretrieved at 1510 to its cache and at 1508 increments (to a value of“1”, e.g.) a reference counter associated with the newly-cached object.

FIG. 16 illustrates an embodiment of a process used to purge cachedcopies of objects that are no longer in use by any clients. At 1602 anindication is received from an associated client context that it nolonger needs a particular object. At 1604 the reference counterassociated with the object specified in 1602 is decremented. At 1606 itis determined whether the reference counter is greater than zero. If itis determined at 1606 that the reference counter is not greater thanzero, it is concluded that no client contexts are using the object, andthe object is purged from cache at 1608. If it is determined at 1606that the reference counter is greater than zero, then it is concludedthat at least one client context is using the object, and, therefore,the object is maintained in cache.

FIG. 17 illustrates an embodiment of a process used to synchronize acached copy with its associated persisted data, if appropriate. At 1702a save request regarding an object is received from an associatedclient. At 1704 the cached copy of the object requested to be saved in1702 is synchronized with the associated data as it is to be saved perthe save request received in 1702 (i.e. the changes are saved to thecached copy), if appropriate (e.g., if the cached copy is not to bepurged). At 1706, the store object of the destination store of theobject requested to be save at 1702 is used to save the object to itsassociated store.

FIG. 18 illustrates an embodiment of an environment in which thepersistent store coordinator manages communication between a clientcontext and available persistent data stores, one of which is an“in-memory” store. In some embodiments, an in-memory store is used toenable a client context to temporarily store data that is not associatedwith a persistent data store accessible via a PSC associated with theclient context and in some embodiments make such data available to otherclient contexts via a PSC. As depicted in environment 1800, thefunctionality of an in-memory store 1802 is the same as persistent datastores, such as SQL database 1804 and XML file 1806. The only differencebetween an in-memory store and other stores is that the in-memory storeis “in-memory” and is not written to disk, removable storage media, etc.by operation of the PSC (i.e., the PSC is not involved in storing thedata persistently). When a client context (e.g., 1808) requests itsassociated persistent store coordinator (e.g., 1810) to add an in-memorystore, the client context provides the persistent store coordinator withthe configuration of the in-memory store, as it does when adding anyother persistent data store. However, because the store is “in-memory”,the client context does not need to specify a URL designating thelocation of the store when requesting the persistent store coordinatorto add the store. In some embodiments, the in-memory store may never bepersistently stored. In some embodiments, however, a client contextmaintains its own persistent data store (e.g. client database 1812).Such an “in memory” store enables a client context to use its ownproprietary data format (or other format not supported by the PSC) topersistently store the data temporarily stored in the in-memory store.The in-memory store allows a client context to make use of thefunctionality of environment 1800 while allowing the client context tomaintain its custom data format. In some embodiments, an “in memory”store may be used as a “scratch pad” shared by two or more clientcontexts.

FIG. 19 illustrates an embodiment of a process used to add a store,including an in-memory store. At 1902 a request to add a store isreceived from a client context. It is determined at 1904 whether thetype of store requested at 1902 to be added is “in-memory”. If it isdetermined at 1904 that the type of the store requested at 1902 to beadded is not “in-memory”, at 1906 a store object is instantiated and amapping is associated with the store as in process 800 of FIG. 8. If itis determined at 1904 that the type of the store requested to be addedat 1902 is “in-memory”, at 1908 an “in-memory” store object isinstantiated. At 1910, memory is allocated for the in-memory store. At1912, data is received from the client context and added to thein-memory store. The data then becomes available via the PSC to theclient context that provided the data and, if configured, to otherclient contexts associated with the PSC.

The approaches described herein enable one or more client contexts to beprovided with access to a potentially dynamically changing set ofpersistent data stores, and data from multiple persistent stores may beaccessed by and/or presented to one or more client contexts as if from asingle data source. Using the approaches described above simplifies thedevelopment of client application programs and provides continuousaccess to those persistent data stores that remain available even asdata stores are added or removed, expectedly or otherwise.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

1. A method for managing objects from multiple persistent stores,comprising: associating one or more data objects with a client context;retrieving the one or more data objects from two or more persistent datastores; providing the one or more data objects to the client contextincluding by making it appear to the client context that the one or moredata objects come from a single source; and providing to the clientcontext a update to the one or more data objects in the event apersistent data store included in the two or more persistent data storesbecomes unavailable or in the event a new persistent data store isadded; wherein the update is provided in real time without affecting theavailability to the client context of any data object not affected by apersistent data store that has become unavailable or a new persistentdata store that has been added.
 2. A method as recited in claim 1,further comprising: receiving an indication that a removed storeassociated with the plurality of persistent data stores has becomeunavailable; and informing the client context that a removed data objectassociated with the removed store is no longer available.
 3. A method asrecited in claim 2, wherein informing the client context includespruning from an object graph associated with the client context one ormore objects associated with the removed store.
 4. A method as recitedin claim 1, further comprising continuing to make available to theclient context without interruption, in the event that a removed one ofthe one or more of the plurality of persistent data stores becomesunavailable, those ones of the one or more data objects that are notassociated with the removed persistent data store.
 5. A method asrecited in claim 1, further comprising receiving from the client contexta request to retrieve the one or more data objects.
 6. A method asrecited in claim 1, wherein associating one or more data objects with aclient context comprises constructing a model that represents the one ormore data objects and their relationship, if any, to one another.
 7. Amethod as recited in claim 6, wherein constructing a model includesreceiving from the client context a definition of one or more types ofobject associated with the client context.
 8. A method as recited inclaim 1, wherein the plurality of persistent data stores comprises datastores of two or more types and retrieving the one or more data objectsfrom one or more of the plurality of persistent data stores includessending to each of the one or more of the plurality of persistent datastores a query in a form associated with that persistent data store. 9.A method as recited in claim 1, wherein the plurality of persistent datastores comprises data stores of two or more types and the method furthercomprises receiving in a format not specific to any persistent datastore type a request to retrieve the one or more data objects.
 10. Amethod as recited in claim 1, further comprising receiving a request toadd a new persistent data store to the plurality of persistent datastores, adding the new persistent store, and retrieving from thepersistent store one or more additional data objects associated with theclient context.
 11. A method as recited in claim 10, further comprisinginforming the client context of the availability of the one or moreadditional data objects.
 12. A method as recited in claim 10, furthercomprising providing the one or more additional data objects to theclient context.
 13. A method as recited in claim 1, further comprisingreceiving a request to store a new data object persistently, associatingthe new data object with an associated one of the plurality ofpersistent data stores, and storing the new data object in theassociated persistent data store.
 14. A method as recited in claim 13,wherein associating the new data object with an associated one of theplurality of persistent data stores includes receiving data designatingthe associated persistent data store.
 15. A method as recited in claim13, wherein associating the new data object with an associated one ofthe plurality of persistent data stores includes identifyingprogrammatically a persistent data store suitable to store the new dataobject.
 16. A method as recited in claim 13, wherein the associatedpersistent data store comprises a primary persistent data store for thenew object and the method further comprises storing a mirror copy of thenew data object in a second persistent data store.
 17. A method asrecited in claim 16, further comprising updating the mirror copy to keepit in the same state as the new data object as stored in the primarypersistent data store.
 18. A method as recited in claim 1, furthercomprising migrating one or more of the one or more data objects from afirst persistent data store to a second persistent data store andcontinuing to make the migrated one or more data objects available tothe client context.
 19. A method as recited in claim 18, furthercomprising informing the client context that the migrated one or moredata objects are now stored in the second persistent data store.
 20. Amethod as recited in claim 18, wherein the migration is performed in amanner that is transparent to the client context and the migrated one ormore data objects remain available to the client context after themigration in the same manner as before the migration.
 21. A method asrecited in claim 1, further comprising storing in a cache a cache copyof one or more of the retrieved one or more data objects.
 22. A methodas recited in claim 21, further comprising incrementing a referencecounter each time a cached data object is requested.
 23. A method asrecited in claim 21, further comprising decrementing a reference counterassociated with a cached data object each time an using client contextassociated with the cached data object indicates it no longer needs thecached data object.
 24. A method as recited in claim 23, furthercomprising purging the cached copy from the cache if the referencecounter as decremented is not greater than zero.
 25. A method as recitedin claim 21, further comprising maintaining the cache copy in synch withthe data object with which it is associated by updating the cache copyas changes to the associated data object are saved.
 26. A method asrecited in claim 1, wherein retrieving the one or more data objects fromone or more of a plurality of persistent data stores includesdetermining whether a previously-retrieved copy of one or more of theone or more data objects is stored in a cache.
 27. A method as recitedin claim 1, further comprising saving in each persistent data store, ina manner not specific to any particular type of persistent data store,metadata associated with the persistent data store.
 28. A method asrecited in claim 27, further comprising associating with the metadatadata indicating a type of persistent data store.
 29. A method as recitedin claim 27, wherein the metadata includes data associated with one ormore data objects stored in the persistent data store.
 30. A method asrecited in claim 27, wherein retrieving the one or more data objectsfrom one or more of a plurality of persistent data stores includesidentifying the one or more persistent data stores based at least onmetadata associated with the one or more persistent data stores.
 31. Amethod as recited in claim 27, wherein retrieving the one or more dataobjects from one or more of a plurality of persistent data storesincludes identifying the one or more persistent data stores based atleast on configuration data provided by the client context wherein theconfiguration data identifies for each persistent data store associatedwith the client context one or more data objects that are associatedwith the persistent data store.
 32. A method as recited in claim 1,further comprising retrieving one or more of the one or more dataobjects from an in memory data store.
 33. A method as recited in claim32, wherein the client context is one of a plurality of client contextsand the method further comprises making data objects stored in the inmemory data store available to two or more of the plurality of clientcontexts.
 34. A system for managing objects from multiple persistentstores, comprising: a processor configured to: associate one or moredata objects with a client context; retrieve the one or more dataobjects from two or more persistent data stores; provide the one or moredata objects to the client context including by making it appear to theclient context that the one or more data objects come from a singlesource; and provide to the client context a update to the one or moredata objects in the event a persistent data store included in the two ormore persistent data stores becomes unavailable or in the event a newpersistent data store is added; wherein the update is provided in realtime without affecting the availability to the client context of anydata object not affected by a persistent data store that has becomeunavailable or a new persistent data store that has been added; and amemory coupled to the processor and configured to provide instructionsto the processor.
 35. A system as recited in claim 34, wherein theprocessor is further configured to: receive an indication that a removedstore associated with the plurality of persistent data stores has becomeunavailable; and inform the client context that a removed data objectassociated with the removed store is no longer available.
 36. A systemas recited in claim 35, wherein the processor is configured to prunefrom an object graph associated with the client context one or moreobjects associated with the removed store.
 37. A system as recited inclaim 34, wherein the processor is further configured to continue tomake available to the client context without interruption, in the eventthat a removed one of the one or more of the plurality of persistentdata stores becomes unavailable, those ones of the one or more dataobjects that are not associated with the removed persistent data store.38. A system as recited in claim 34, wherein the processor is configuredto associate one or more data objects with the client context includingby constructing a model that represents the one or more data objects andtheir relationship, if any to one another.
 39. A system as recited inclaim 34, wherein the plurality of persistent data stores comprises datastores of two or more types and the processor is configured to retrievethe one or more data objects from one or more of the plurality ofpersistent data stores including by sending to each of the one or moreof the plurality of persistent data stores a query in a form associatedwith that persistent data store.
 40. A computer program product formanaging objects from multiple persistent stores, the computer programproduct being embodied in a computer readable medium and comprisingcomputer instructions for: associating one or more data objects with aclient context; retrieving the one or more data objects from two or morepersistent data stores; providing the one or more data objects to theclient context including by making it appear to the client context thatthe one or more data objects come from a single source; and providing tothe client context an update to the one or more data objects in theevent a persistent data store included in the two or more persistentdata stores becomes unavailable or in the event a new persistent datastore is added; wherein the update is provided in real time withoutaffecting the availability to the client context of any data object notaffected by a persistent data store that has become unavailable or a newpersistent data store that has been added.
 41. A computer programproduct as recited in claim 40, further comprising computer instructionsfor: receiving an indication that a removed store associated with theplurality of persistent data stores has become unavailable; andinforming the client context that a removed data object associated withthe removed store is no longer available.
 42. A computer program productas recited in claim 41, wherein informing the client context includespruning from an object graph associated with the client context one ormore objects associated with the removed store.
 43. A computer programproduct as recited in claim 40, further comprising computer instructionsfor continuing to make available to the client context withoutinterruption, in the event that a removed one of the one or more of theplurality of persistent data stores becomes unavailable, those ones ofthe one or more data objects that are not associated with the removedpersistent data store.
 44. A computer program product as recited inclaim 40, wherein associating one or more data objects with a clientcontext comprises constructing a model that represents the one or moredata objects and their relationship, if any, to one another.
 45. Acomputer program product as recited in claim 40, wherein the pluralityof persistent data stores comprises data stores of two or more types andretrieving the one or more data objects from one or more of theplurality of persistent data stores includes sending to each of the oneor more of the plurality of persistent data stores a query in a formassociated with that persistent data store.